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ABSTRACT 



This speech highlights some of the problems of. 
translating research findings, regardless of the research methodology 
used, into educational policy at the national level. The author 
discusses the problems of (1) the division of responsibility for 
education among Federal, State, and local governments; (2) the lack 
of consensus about educational objectives; (3) the policymakers* lack 
of faith in new research findings; and (4) the rapid pace of change 
in the world today. He suggests some ways for alleviating these 
problems. (JF) 
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The preceding speakers have outlined problems associated with 
nation-wide cross-sectional surveys, longitudinal surveys, and experimental 
interventions. Each of the three types of research has the objective of 
guiding educational policy. Our task is to highlight some of the problems 
bearing directly on the complicated process of translating research find- 
ings, regardless of the research methodology used, into educational policy 
at the national level. The problems we shall outline are neither new to 
the American political scene nor unique to education, but they are perhaps 
more pronounced in the field of education than in other policy areas. 

The first problem, perhaps too obvious to mention but too obtrusive 
to ignore, is the severely restricted and still unsettled boundary of 
federal government responsibility in the area of education. Sharing of 
responsibility with state and local government is not uncommon in our 
federal system, of course, although researchers and policy analysts from 
other government agencies dealing vrith defense or international affairs 
or postal services or atomic energy, etc., may tend to underestimate the 
problem in such areas as education. But the problem isn't simply a matter 
of divided responsibility. In the area of education there is little agree- 
ment about the proper lines of division. The lines dividing authority 
are vaguely drawn and hotly contested. There have been controversies over 
boundaries, and significant redrawing of boundaries, in many policy areas 
in the past: anti-trust, labor relations, banking, technological research 

and development, agriculture, to name just a few. But it seems fair to 
say that the controversy over boundaries of responsibility in education 
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has persisted over a longer time, and has generated more intense feelings, 
and has involved more people and more levels of government, than any of the 
others . 

Hew do we translate research findings into policy in the midst of a 
persistent tug-of-war over policy responsibilities? There is an easy 
answer to this question and a more challenging answer. We believe both 
answers have merit. 

The easy answer is to suggest that a researcher is interested in 
truth, not policy, and that he could care less whether educational policy 
is going to be made by the federal executive, or by the courts, as may be 
the case, or by states, or by school boards, or by administrators or 
teachers or students or parents, or by schools of education, or by some 
complex and ever-changing combination of all of these. But this answer 
has its limitations. First, some researchers are interested in policy as 
well as truth. Second, and more important, those who allocate funds for 
educational research are very definitely interested in policy as well as 
truth. And if research is to have direct implications for policy at the 
federal government level, then the divisions of responsibility must be 
taken into account. Most particularly, the reactions of other policy 
makers to any proposed federal policy must be 4 taken into account. We make 
no apology for emphasizing this trite and obvious fact because it is often 
forgotten as quickly as it is admitted. 

If we remembered the division of responsibility, and if we were 
seriously engaged in the analysis of policies, then we would recognize 
the need to estimate carefully the reaction of other policy makers to any 
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when they stump and thump for education, though that would be problem 

enough; they have directly conflicting things in mind. We couldn’t even 

agree on whether the goal of education is to develop the intellect, or to 

inculcate values, or to c.mmunicate knowledge, or to encourage conformity, 

or to raise aspirations, or to discipline, or to preoccupy, or to generate 

confidence, or to give joy, or to run an ever-lengthening competition for 

% 

job opportunities — and any one of these objectives is ambiguous enough 
to contain a host of disagreements within itself. 

How are we going to translate research into policy when there is 
nothing but controversy over policy objectives? The educational researcher 
can choose among several typical responses to this problem, none of them 
being entirely satisfactory. An easy response, as we suggested earlier, is 
to decide that he is interested only in truth, not in policy. A second 
response is to insist that the controversy over educational goals must be 
resolved as the first order of business. There is a measure of logic in 
this response, of course, as well as a large measure of futility. Even the 
youngest of us here today won’t be a witness to the resolution of this 
conflict. Incidentally, we don’t cite this difficulty of achieving consensus 
as an indication of the cantankerous quality of the human species. There 
is no reason why people should ever agree on educational goals. 

A third response to. the problem is to insist that some authority 

t 

should list goals and priorities, arbitrarily if need be, for guiding 
research and policy analysis . This may sound like a reasonable, pragmatic 
solution to the analyst’s needs, but it has its limitations. 

We all know that a listing of vague and ambiguous goals is not too 
helpful. But a listing of very precise goals is net always helpful either. 
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Even if different goals in the list are not directly conflicting, they 
are more than likely to be competitive. Certainly the pursuit of any one 
goal will use time and attention and resources that might have been 
devoted to the pursuit of other goals. If an analyst doesn’t know this 
simple fact of life at the outset of his research he will discover it 
soon enough, and then he may think the problem can be solved by assigning 
priorities. He is still deluded, though this is a common enough delusion. 

We haven’t really answered very many practical issues by deciding 
to list one educational objective first, another second, another third, 
and so on. We aren’t going to direct all of our efforts toward the 
attainment of the first priority objective, nor even the top two or three 
objectives. We are going to compromise them all and distribute our efforts 
over a broad range. The challenge facing policy makers is to decide how 
to distribute the effort, and the challenge of policy-relevant research is 
to help guide that decision. 

We can summarize this second point we are making by saying that, 
if research is to have direct implications for educational policy, then 
researchers should neither attempt to evade the question of educational 
objectives nor should they adopt simple answers to the question. Instead, 
more effort should be expended to satisfy these two needs: first, the 

need to find out what different educational objectives people have in mind, 
and how strongly they support these objectives, and why; and second, the 
need to find out how much of any one objective must be sacrificed in order 
to pursue any other objective. These are researchable questions, requiring 
data, not arbitrary value judgments, and they are questions for which 
policy-makers need answers. 



A third problem associated with nation-wide evaluation and the 
formulation of educational policy is the problem of confidence. Given 
the current state-of-the-arts in educational research, and given our 
current limited knowledge about educational effects, there is a certain 
reluctance on the part of many policy makers to place implicit faith in 
new research findings. Even when we learn something important — or when 
we think we have learned something important — we are frequently met 
with widespread skepticism. ? 

It is time that policy makers dismiss most quickly those findings 
which they don't like. And it is true that policy makers will accept 
many other findings which appeal to them regardless of the strength of 
the evidence. Policy makers are like that. Parents and teachers and 
students and administrators are like that, and perhaps educational 
researchers too. Ambiguity of findings is always taken by some people 
as an excuse to believe what they want to believe. 

This problem is not at all unique to the educational policy arena. 

It exists in the arenas of public welfare, international relations, ecology, 
health, and any other you may name. Educational researchers can alleviate 
the problem by becoming familiar with the common rationalizations of policy 
makers and anticipating their diversionary reactions to important research 
findings ana marshalling evidence to head off such diversions. 

Since the educational system is vastly complex and since no two 
students or teachers or classrooms are alike, the number of things we know 
with absolute certainty about education is rather limited. Policies 
must, inevitably, be guided by rough estimates as well as by precise measures, 
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and by reasonable hypotheses as well as by confirmed theories. But once 
we leave the bedrock of well-tested theories and start building our 
educational system on less solid ground, we sometimes seem to lose touch 
with the ground altogether. 

Perhaps a practice of our educational system itself is at fault here. 
W.e try to teach students to distinguish what they know from what they don't 
know, and while that sounds like a good idea indeed, perhaps we unwittingly 
neglect important matters of degree. If we tried to divide up the realm 

n 

of educational policy questions into two categories, those questions we 
really know the answers to and those we know nothing about at all, there 
wouldn’t be many questions in either category. There is, after all, a middle 
ground between what we fully comprehend and what completely mystifies us. 
There are educational hypotheses for which there is persuasive, but not 
compelling, evidence. For the next half century at least, educational 
policies are going to be made on the basis of what we think, not what we 
know ; that is, on the basis of judgments. Obviously we shall be more con- 
fident of some of our judgments than we are of others and that forces us 
to deal witn the matter of degrees of confidence. 

We are not concerned here simply with computing standard errors of 
estimate or confidence intervals or the like, although that is a gesture 
in the right direction. We are concerned with far more difficult questions - 
For example, how much confidence should a policy maker have that surveys 
of first graders in 1972 have valid implications for first graders in 1982? 
How much confidence should a policy maker have that findings of an experiment 
under certain controlled conditions will apply in somewhat altered conditions? 
How much confidence should he have that observed short-run efforts of an 
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educational innovation won't be overbalanced by unobserved long-run effects. 
There is absolutely no way to translate research into policy without making 
a host of difficult judgments such as these. And that is where the 
controversies, the uncertainties, the confusions, and the disappointments 
come into the formulation of educational policy. 

We may summarize this third point by noting that educational policy 
makers need more than raw research data as grist for the policy-making mill. 
They need confidence aopraisals of research findings, and confidence 
appraisals of the many possible interpretations and policy implications 
that are drawn from the findings. Supplying these appraisals is a 
challenging task deserving as much effort, and ingenuity, and care, and 
quality control, and criticism as the task of conducting basic research 
itself. We believe that the challenge receives only haphazard attention 
and that the research community as a whole should undertake a major respon- 
sibility for bringing more systematic analysis to the problem. 

We shall mention but one more problem concerning nation-wide 
evaluation and the formulation of educational policy, and that has to do 
with the fact that the world is changing. We doubt that any of you will 
challenge this assertion. 

Change at a slow pace is only a problem for the disadvantaged, but 
change at a rapid pace can be a problem for everyone, including the policy 
makers and researchers. Policy makers can adapt to slow changes by waiting 
for problems to manifest themselves; by making incremental changes in laws 
and budgets and programs; and by testing these incremental changes in 
actual use, revising them as necessary. Researchers can adjust to slow 
changes by making incremental revisions in their assumptions and by 
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gradually shifting the focus of their attention. Rapid change is likely 
to overwhelm these incremental adaptive processes and leave both policy 
makers and researchers hopelessly outdated. 

We don't intend to dwell on the many obvious problems confronting 
the researcher as he tries to keep up with the changing world. Instead, 
we want to point -out that the .challenge of translating research into 
policy takes on a very different character once we begin to modify our 
customary incrementalist policy-making style in response to the pressures 
generated by rapid change. We will mention two modifications in policy- 
making style and note the special problems they" will generate for the 
research community. 

The first modification in policy-making style is in the direction 
of longer lead times and more comprehensive policies. Faced with increasingly 
rapid change the policy makers discover that they need to focus on the 
future, not the present, and they discover that the solution to developing 
problems often require integrated and complex packages of changes rather 
than a few isolated incremental changes. In other words, the policy makers 
will start searching for alternative blueprints for the future as opposed 
to minor modifications of the present. 

• We may have the best education research in the world, but we aren't 
going to find it easy to supply the blueprints that the policy makers want. 

It is difficult enough to alter some single variable in the current educational 
scene and to observe and measure its effects. If the policy makers want to 
know what will happen if a dozen important variables are altered simultaneous- 
ly, who is going to tell them? Yet, they need to know! The capability of 
our society to cope with the increasingly rapid rate of change may hinge upon 
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the capacity of our researchers to predict the consequences of numerous 
simultaneous changes - 



If the challenge of designing and orchestrating comprehensive 



changes seems too great, policy making styles can be modified in a different 
direction. We can open up opportunities for widespread innovation and 
variation, and we can attempt to manipulate the incentive structure so as 
to encourage promising developments. This is the policy style that has 
been most prominent in this country in the economic sphere. This policy 
style imposes different demands on the educational research community. 

The need here is for assessment of the impact of truly decentralized decision 
making, and of wide variations in educational practice, and of continuous 
change, and of coordinated modifications of the complex of incentives 
operating on -the system. It is a different style of educational research 
than is required for the development of alternative blueprints, and either 
of these styles is different from that characteristic of most of today's 
educational research. 
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